Skip to content
This repository has been archived by the owner on Jul 24, 2024. It is now read-only.

lightning: save chunk checkpoint timely #1080

Merged
merged 12 commits into from
Jun 7, 2021
Merged

lightning: save chunk checkpoint timely #1080

merged 12 commits into from
Jun 7, 2021
+257 −71

Conversation

glorv
Copy link
Collaborator

@glorv glorv commented Apr 30, 2021 •
edited
Loading

What problem does this PR solve?

Try to save chunk checkpoint timely.

What is changed and how it works?

  • Return a chunk status object when close chunk but it's not flushed. Then before each chunk restorer started, check and save all the pending chunk checkpoint if available.
  • Try to save the chunk checkpoint before and after each deliver loop.

Check List

Tests

  • Unit test
  • Integration test

Code changes

  • Has interface methods change

Side effects

  • Possible performance regression
  • Increased code complexity

Related changes

  • Need to cherry-pick to the release branch
  • Need to update the documentation

Release Note

  • check and save chunk checkpoint timely.

@ti-chi-bot ti-chi-bot requested a review from YuJuncen April 30, 2021 10:03
@glorv
Copy link
Collaborator Author

glorv commented May 6, 2021

/run-all-tests

@ti-chi-bot ti-chi-bot added size/XL and removed size/L labels May 6, 2021
@glorv
Copy link
Collaborator Author

glorv commented May 6, 2021

/run-all-tests

@glorv
Copy link
Collaborator Author

glorv commented May 6, 2021

/run-all-tests

1 similar comment
@glorv
Copy link
Collaborator Author

glorv commented May 6, 2021

/run-all-tests

@glorv glorv marked this pull request as ready for review May 6, 2021 09:31
@glorv glorv requested review from Little-Wallace and kennytm May 6, 2021 09:31
@glorv
Copy link
Collaborator Author

glorv commented May 6, 2021

/run-all-tests

@glorv
Copy link
Collaborator Author

glorv commented May 7, 2021

/run-all-tests

@glorv
Copy link
Collaborator Author

glorv commented May 31, 2021

/run-all-tests

1 similar comment
@glorv
Copy link
Collaborator Author

glorv commented Jun 1, 2021

/run-all-tests

@glorv glorv added this to the v5.1.0 milestone Jun 2, 2021
@glorv
Copy link
Collaborator Author

glorv commented Jun 2, 2021

/run-all-tests

@glorv
Copy link
Collaborator Author

glorv commented Jun 4, 2021

/run-all-tests

@ti-chi-bot
Copy link
Member

@gozssky: Thanks for your review. The bot only counts approvals from reviewers and higher roles in list, but you're still welcome to leave your comments.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Member

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • Little-Wallace
  • kennytm

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added status/LGT2 LGTM2 and removed status/LGT1 LGTM1 labels Jun 7, 2021
@glorv
Copy link
Collaborator Author

glorv commented Jun 7, 2021

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: e4924eb

@ti-chi-bot ti-chi-bot merged commit 37433a1 into pingcap:master Jun 7, 2021
@glorv glorv deleted the chunk-cp branch June 7, 2021 14:34
ti-chi-bot pushed a commit to ti-chi-bot/br that referenced this pull request Jun 7, 2021
@glorv @ti-chi-bot
This is an automated cherry-pick of pingcap#1080
ee6de9c 
Signed-off-by: ti-chi-bot <[email protected]>
@ti-chi-bot ti-chi-bot mentioned this pull request Jun 7, 2021
Closed
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #1181.

ti-chi-bot pushed a commit to ti-chi-bot/br that referenced this pull request Jun 7, 2021
@glorv @ti-chi-bot
This is an automated cherry-pick of pingcap#1080
b327f77 
Signed-off-by: ti-chi-bot <[email protected]>
@ti-chi-bot ti-chi-bot mentioned this pull request Jun 7, 2021
Closed
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #1182.

@ti-chi-bot ti-chi-bot mentioned this pull request Jun 7, 2021
Merged
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #1183.

ti-chi-bot pushed a commit to ti-chi-bot/br that referenced this pull request Jun 7, 2021
@glorv @ti-chi-bot
lightning: save chunk checkpoint timely (pingcap#1080)
51b0042
ti-chi-bot added a commit that referenced this pull request Jun 10, 2021
@ti-chi-bot
lightning: save chunk checkpoint timely (#1080) (#1183)
546ae08
@glorv glorv mentioned this pull request Jun 16, 2021
Merged
11 tasks
@sleepymole sleepymole mentioned this pull request Jun 28, 2021
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@overvenus overvenus overvenus left review comments

@sleepymole sleepymole sleepymole approved these changes

@Little-Wallace Little-Wallace Little-Wallace approved these changes

@kennytm kennytm kennytm approved these changes

@YuJuncen YuJuncen Awaiting requested review from YuJuncen

Assignees
No one assigned
Labels
component/import needs-cherry-pick-release-4.0 needs-cherry-pick-release-5.0 needs-cherry-pick-release-5.1 size/XL status/can-merge status/LGT2 LGTM2 status/ptal
Projects
None yet
Milestone
v5.1.0
Development

Successfully merging this pull request may close these issues.
7 participants
@glorv @3pointer @sleepymole @ti-chi-bot @kennytm @overvenus @Little-Wallace